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Abstract — Recent work characterizing tlie optimal per- 
formance of secrecy systems lias made use of a distortion- 
like metric for partial secrecy as a replacement for the 
more traditional metric of equivocation. In this work we use 
the log-loss function to show that the optimal performance 
limits characterized by equivocation are, in fact, special 
cases of distortion-based counterparts. This observation 
illuminates why equivocation doesn't tell the whole story 
of secrecy. It also justifies the causal-disclosure framework 
for secrecy (past source symbols and actions revealed to 
the eavesdropper). 

Secrecy systems have a dual purpose — to reliably 
communicate to an intended receiver while hiding the 
information from an eavesdropper Reliable commu- 
nication is typically measured operationally — i.e. the 
reconstruction of the information content must have a 
negligible probability of error, or when perfect transmis- 
sion is not possible, a distortion constraint is met. On the 
other hand, the conventional approach for information 
theoretic secrecy is to measure the secrecy aspect of the 
communication using equivocation (the posterior entropy 
of the information signal), rather than an operational 
quantity. In this case, entropy and mutual information are 
a part of the problem statement itself, rather than being 
unexpectedly discovered in the answer Thus, this line 
of research forfeits one of the most elegant trademarks 
of information theory. 

There are two regimes of interest in information 
theoretic secrecy, the perfect secrecy regime, and the 
partial secrecy regime. The perfect secrecy case, where 
no information is leaked to the eavesdropper, is where 
the subtleties of "strong secrecy" versus "weak secrecy" 
(whether or not the leaked mutual information is neg- 
ligible before or after being normalized by n) enters 
the conversation. That is not the regime of interest in 
this paper. This paper is focussed mainly on the partial 
secrecy case. 

The use of equivocation "as a measure of the degree to 
which the wire-tapper is confused" can be found at least 
as early as Wyner's work on the wiretap channel [1]. A 
direct reference to this standard was made by Csiszar 
and Komer in their related work in [2], where they also 
use normalized equivocation to quantify partial secrecy. 
This trend continues today, as is evident in these recent 



contributions: [3], [4], [5], [6]. It is true that Shannon's 
much earlier 1949 paper on secrecy systems [7] also 
refers to equivocation, but in his case it was simply 
used as an analysis tool, not a definition of secrecy — 
the extreme values of equivocation correspond to perfect 
secrecy and error-free decoding, both of which can be 
defined with simple, operational statements. 

In [8] and [9], we proposed a rate-distortion approach 
to measuring partial secrecy. The communication system 
is designed to guarantee a certain level of distortion 
in the eavesdropper's reconstruction of the information 
source. It was discovered that the richest theories emerge 
when the eavesdropper is empowered by having causal 
access to either the information signal or the intended 
receiver's reconstruction or both in addition to eaves- 
dropping on the communication. Without this causal 
disclosure of information, the communication system can 
get away with inexpensive tricks to keep the eavesdrop- 
per's distortion high even while the eavesdropper has 
very little uncertainty about the information signal [10]. 

The causal disclosure of information to the eaves- 
dropper in [8] and [9] plays a fundamental role in the 
results, yet it is an uncommon assumption in commu- 
nication systems. An operational justification for causal 
disclosure is that the eavesdropper is actively trying to 
disturb the system — the distortion function measuring 
the eavesdropper's performance can be generalized to 
be the payoff of a zero-sum repeated game between the 
eavesdropper and the system. In a repeated game context, 
disclosure of the past actions in the game can be a natural 
assumption. However, the results of this work provide 
an entirely different justification for the rate-distortion- 
with-causal-disclosure framework for measuring partial 
secrecy. 

It turns out that the equivocation approach to measur- 
ing partial secrecy and the rate-distortion approach are 
not in fact two different approaches. The rate-distortion 
approach provides the general theory, yielding the opti- 
mal equivocation characterization as a special case. In 
fact, equivocation is one of the most blunt special cases 
of the theory in the sense that it requires the least care in 
designing the optimal communication system. The idea 
of this work is very simple. We apply the theorems for 



rate-distortion theory with causal disclosure in [9] to the 
log-loss distortion function. This technique immediately 
generalize to any secrecy setting. That is, once distortion- 
based secrecy with causal disclosure is understood for 
a secrecy setting, then the characterization of optimal 
equivocation can be readily obtained. 

L Rate-distortion theory with causal 

DISCLOSURE 

Consider a cipher system with an i.i.d. information 
source, Xi ^ Px- Secret key is available at a rate of 
Rq bits per source symbol to the encoder A and the 
decoder B. The encoder transmits a message M at rate 
R bits per source symbol to the decoder which is also 
received by the eavesdropper The decoder then produces 
a sequence Yi, the eavesdropper produces a sequence 
Zi, and the performance of the system is measured by a 
payoff function n{Xi, Yi, Zi) 

From [9] we obtain theorems that give so-call "single- 
letter" information theoretic characterizations of the fol- 
lowing three quantities for any payoff function n and 
source distribution Px- 
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The difference between these three quantities is the 
information that the Eavesdropper has available when 
forming Zi. 

II. Log-loss 
Consider three log-loss payoff functions 
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where x and y have domains X and y, and z is a 
probability mass function on the space X for tti, the 
space y for tt2, and the space X x y for tt^. In other 
words, z(a) > for all a, and X^a -^('^) ~ ^■ 

Log-loss functions have been studied as distortion 
functions for multi terminal source coding in [11] and 
[12], in which the authors refer to the use of log- 
loss functions in the study of information bottleneck in 
[13] and as a distortion measure in image processing 
[14]. Our reason for analyzing communication under 



this payoff (or "distortion") function is the following 
convenient property: 

min F,TTi{X,Y,Z) = ■ - • 1 
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where X, Y, and U are arbitrarily correlated. Likewise, 

min 'Ett2(X,Y,Z) = H(Y\U), 

Z=z{U) 

min E7r3(X,y,Z) = H{X,Y\U). 
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III. Equivocation as a Special Case 

We now specialize the rate-distortion theory for se- 
crecy systems with causal disclosure to the case of log- 
loss payoff functions. For example, consider substituting 
TTi as the payoff function in ([T]i- 
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This quantity defines the optimal normalized equivoca- 
tion (with respect to the information source X) that a 
cipher can achieve. We can then apply the information 
theoretic theorems of [9] to obtain a characterization of 
this quantity — the optimal equivocation. 

Before we proceed we must point out one subtle 
detail. The log-loss payoff function puts no constraint 
that the intended receiver must produce an output Y that 
is correlated with the information source X. In fact, it 
rewards the system for choosing Y independent of X 
and sending no description of X. In order to impose a 
genuine communication requirement, we add a second 
distortion constraint, aside from the log-loss payoff. The 
constraint is ^^^i^id{Xi,Yi) < D, and the theorems 
of [9] are readily generalized to include more than one 
payoff (or distortion) function. 

This leads us to our resulting statements about equiv- 
ocation in a cipher system. Upon simplifying Theorems 
7.1, 7.2, and 3.1 of [9] using the payoff functions tti, 
Ti2, and vTs we obtain the following theorem (proof of 
simplification omitted): 

Theorem 3.1: For a cipher system with communica- 
tion rate R, secret key rate Rq, source distribution Px, 
and satisfying an average distortion constraint of D at the 
intended receiver (as measured by d{x, y)), the following 
three statements hold: 



Equivocation of the information source: 

sup -i/(X"|M) = H{X) -[R{D) - Ro] + . 
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Equivocation of the reconstruction: 
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Equivocation of the source and reconstruction: 
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Here [•]+ = max{0, •}, R{D) is the rate-distortion 
function for distortion d and source distribution Px, and 
V is the set of random variable triples that satisfy 

X — U — Y form a Markov chain, 

E d{X, Y) < D, 
I{X;Y) < R. 

IV. Summary 

Information theory allows us to understand the optimal 
use of communication resources not only in the ideal 
settings of lossless compression, noise free communica- 
tion channels, or perfect secrecy, but also in the imper- 
fect settings. Rate-distortion theory helps us understand 
optimal partial communication. Similarly, we endeavor 
to understand partial secrecy, for situations that do not 
permit perfect secrecy. We see from this work that 
a distortion-based measure of secrecy which includes 
causal disclosure of information to the eavesdropper 
(in addition to the intercepted communication signal) 
provides a rich theory for understanding secrecy. This 
theory subsumes the conventional equivocation based 
approach to secrecy in that the latter is a special case 
of the rate-distortion theory via the log-loss function. 

This analysis also highlights the blunt nature of the 
equivocation approach to secrecy. In general, distortion- 
based performance in a secrecy system is optimized by 
a carefully crafted communication scheme. However, 
in the special case of log-loss (i.e. the equivocation 
characterization), many different encoding variations are 
optimal, including in particular random binning or time- 
sharing. An analogy to this occurs in lossy compression. 
If the goal in lossy compression were simply to mini- 
mize the equivocation at the intended receiver, then any 
message transmission would be equally good, as long as 
it is a fully entropic function of the information source. 
But rate-distortion theory reveals much more than this 
about the nature of partial information. 
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